In Regression Testing without Code Dedication

ZHENG, JIANG. In Regression Testing without Code. (Under the direction of Dr. Laurie Williams.) Software products are often built from commercial-off-the-shelf (COTS) components. When new releases of these components are made available for integration and testing, source code is usually not provided by the COTS vendors. Various regression test selection techniques have been developed and have been shown to be cost effective. However, the majority of these test selection techniques rely on source code for change identification and impact analysis. This dissertation presents a regression test selection (RTS) process called Integrated Black-box Approach for Component Change Identification (I-BACCI) for COTS-based applications. The I-BACCI process reduces the test suite based upon changes in the binary code of the COTS component using the firewall analysis regression test selection method. This dissertation also presents Pallino, the supporting automation that statically analyzes binary code to identify the code change and the impact of these changes. Based on the output of Pallino and the original test suit, testers can determine the regression test cases needed that execute the application glue code which is affected by the changed areas in the new version of the COTS component. Five case studies were conducted on ABB internal products written in C/C++ to determine the effectiveness and efficiency of the I-BACCI process. The total size of application and component for each release is about 340~930 KLOC. The results of the case studies indicate this process can reduce the required number of regression test by as much as 100% if there are a small number of changes in the new component in the best case. Similar to other RTS techniques, when there are many changes in the new component the I-BACCI process suggests a retest-all regression test strategy. With the help of Pallino, RTS via the I-BACCI process can be completed in about one to two person hours for each release of the case studies. Depending upon the percentage of test cases reduction determined by the I-BACCI process, the total time cost of the whole regression testing process can be reduced to 0.0003% of that by retest-all strategy in the best case. Pallino is extensible and can be modified to support other RTS methods for COTS components. Currently, Pallino works on components in Common Object File Format or Portable Executable formats. © Jiang Zheng 2007 All Rights Reserved In Regression Testing without Code By Jiang Zheng A dissertation submitted to the Graduate Faculty of North Carolina State University in partial fulfillment of the requirements for the Degree of Doctor of Philosophy